Joint and Discriminative Dictionary Learning for Facial Expression Recognition

Dictionary Learning and sparse coding methods have been widely used in computer vision with applications to face and object recognition. A common challenge when performing expression recognition is that face similarities may confound the expression recognition process. An approach to deal with this problem is to learn expression specific dictionaries, so that each atom corresponds to one expression class. However, even when employing expression specific dictionaries, it is likely that two atoms from two sub-dictionaries share common characteristics due to facial similarities. In this paper, we consider a joint dictionary that captures common facial attributes, and class-specific dictionaries that are used to classify different expressions. We investigate three dictionary learning methods for sparse representation classification: one that learns a global dictionary based on K-SVD, one that learns expression specific dictionaries based on Fisher Discrimination Dictionary Learning (FDDL), and one that learns a shared as well as expression specific dictionaries based on Dictionary Learning Separating Commonality and Particularity (DL-COPAR). We demonstrate the effectiveness of the shared dictionary learning approach on the extended CohnKanade database where DL-COPAR outperforms FDDL and KSVD by a significant margin. Introduction Facial expression recognition has many applications such as human-computer interaction, driver monitoring, health and wellness, entertainment, surveillance and others. Recognizing facial expressions is a challenging task, and sometimes similarities in facial appearance may interfere with the recognition of facial expressions. In this paper, we propose a sparse representation classification approach with joint and discriminative dictionary learning in order to overcome the difficulty of confounding face expression and identity. The pioneering work of Ekman et al. [1], identified six universal expressions shown in Figure 1, and introduced a method to quantify facial actions and expressions based on action units. The Facial Action Coding System (FACS) was proposed to quantify facial actions based on muscle movements, so that each expression can be represented as a combination of action units. Numerous facial expression recognition methods have been presented in the literature [2, 3, 4]. These methods can be broadly categorized into geometric and appearance based. Common geometric methods include Active Shape Model (ASM) or Active Appearance Model (AAM) [5]. Appearance based methods work with local or holistic facial appearance. They often compute intermediate representations of images using features such as Gabor wavelets [6] and Local Binary Patterns (LBP) [7]. Gabor wavelets generate features that correspond to edges at various frequencies and orientations inspired from the human visual system. LBP features capture texture variations and are capable of handling severe changes in illumination. Most of the expression recognition pipelines begin high dimensional representations of facial features, and use dimensionality reduction techniques such as Principal Component Analysis (PCA) and manifold learning. Dimensionality reduction benefits the classification process by reducing the data size and organizing the data in a space that improves classification accuracy. Manifold learning techniques have been utilized for expression recognition [8] among other facial analysis tasks. Sparse Representation (SR) classification techniques have demonstrated good performance in face recognition [9] and expression recognition [10], [11]. Manifold based Sparse Representation (MSR) [11] combines manifold learning and sparse representations to tackle the problem of coefficient contamination due to facial identity in expression recognition. Recent developments in dictionary learning methods have shown that learning a dictionary from data is beneficial because it produces better and more efficient representations [12, 13]. In [14], discriminative dictionary learning is proposed by using the class label information. In [15], the authors proposed a discriminative approach that exploits the coherence between atoms in the dictionary to learn a shared/common dictionary and class-specific dictionaries. Other dictionary learning methods include [16-18]. A joint/common dictionary is considered to address the issue of similarities in elements across dictionaries. The proposed approach is effective for dealing with a common problem in expression recognition where the learned system classifies faces that are similar in appearance rather than classifying the expression. In this context, learning an expression specific and a shared dictionary plays an important role in classifying expression with high accuracy. By detecting shared features, we learn subdictionaries whose atoms are not correlated with other dictionaries. Figure 1. Sample images from the extended Cohn-Kanade (CK+) facial expression dataset illustrating (top to bottom, left to right) anger, disgust, fear, happy, sad, surprise. In the next sections, dictionary learning is overviewed for sparse representation classification using K-SVD [12], Fisher Discrimination Dictionary Learning (FDDL) [14], and Dictionary Learning Separating Commonality and Particularity (DL-COPAR) [15]. Results are reported on the extended Cohn-Kanade (CK+) database. ©2016 Society for Imaging Science and Technology DOI: 10.2352/ISSN.2470-1173.2016.11.IMAWM-455 IS&T International Symposium on Electronic Imaging 2016 Imaging and Multimedia Analytics in a Web and Mobile World 2016 IMAWM-455.1 Joint Dictionary Learning Sparse Representation In a sparse representation framework, a sample y in R! space is represented on a dictionary of samples X ∈ R!×! via the sparse coefficients a, as follows: